TITLE OF THE INVENTION 
IMAGE PROCESSING DEVICE, IMAGE PROCESSING METHOD, 
AND IMAGE PROCESSING PROGRAM 

This application is based on Patent Application 
Nos. 2000-267305 and 2001-189865 filed in Japan on 
September 4, 2000, and June 22, 2001, the entire contents 
of which are hereby incorporated by reference. 

BACKGROUND OF THE INVENTION 
FIELD OF THE INVENTION 

The present invention relates to an image 
processing device, image processing method, and image 
processing program, and more specifically relates to 
obtaining specific document data by extracting necessary 
document blocks from image data obtained by reading a 
document such as a newspaper, magazine and the like. 

DESCRIPTION OF THE RELATED ART 

There are times when extraction of data of only 
a specific document is wanted in a document comprising a 
full page surface such as, for example, a newspaper, 
magazine and the like. 

For example, Japanese Laid-Open Patent 
Application No. H9-204511 proposes a device which 



extracts character images of headlines among image data 
after reading a document such as a newspaper, magazine 
and the like to obtain image data, and records the 
associations of the character code data of headlines 
obtained in a character recognition process to the 
extracted character images, and character image data of 
body text corresponding to the headlines. 

Although the device disclosed in this 
publication can obtain character image data of the 
corresponding body text by specifying character code data 
of the headline, disadvantages arise in that the obtained 
document data are difficult to read because the shape of 
the document block (document region) in which the 
document data appears is irregular because the character 
image data of the body text is used directly in the 
layout of the document. Moreover, when the obtained 
document data are adhered to a region of definite form, 
there is much white space since the shape is irregular, 
which is inefficient. 

OBJECTS AND SUMMARY 
An object of the present invention is to 
eliminate the previously described disadvantages by 
providing an improved image processing device, image 
processing method, and image processing program. 



Another object of the present invention is to 
provide an image processing device, image processing 
method, and image processing program capable of 
extracting specific document blocks from among image data, 
5 and obtaining document data readable and effectively 

adherable in a region of definite form without waste. 

These objects are attained by an image 
processing device comprising: 

extraction means for extracting a document 
jjo block containing a specific image from among image data 

' to be processed; ■ - 

Z~i recognition means for recognizing character 

.7* code from the character image within the document block; 

^ reconstruction means for reconstructing the 

"lj5 document block in a specific shape; and 

-II layout means for laying out character code data 

corresponding to character code recognized by the 
recognition means within the reconstructed document block. 

In this image processing device, the extraction 

20 means may extract a plurality of blocks, and the 

reconstruction means may arrange the plurality of 
extracted document blocks into a single block 
reconstructed to a specific shape. 

In this image processing device, the specific 

25 image desirably includes a character image of a headline, 
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and a character image of body text corresponding to the 
headline. In this case it is desirable that headline 
character arrangement means is further provided for 
arranging the character code data corresponding to the 
character image of the headline at a specific position 
within the reconstructed document block. 

In this image processing device, the 
reconstruction means may also adjust a vertical or 
horizontal dimension of the document block to a length 
approximating a natural integer multiple of the length of 
one column of the multiple columns formed within the 
document block. 

This image processing device also may be 
provided with file generation means for generating an 
electronic file storing the character code data laid out 
by the layout means. 

This image processing device also may be 
provided with a printer for printing the character code 
data laid out by the layout means on a recording 
substrate . 

This image processing device also may be 
provided with a reader for reading image data to be 
processed by optically reading an image of a document. 

These objects are further attained by providing 
an image processing method comprising the steps of: 
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extracting a document block containing a 
specific image from among image data to be processed; 

recognizing character code from a character 
image within the document block; 

reconstructing the document block in a specific 

shape; and 

laying out character code data corresponding to 
the recognized character code within the reconstructed 
document block. 

These objects are further attained by providing 
a program for causing a computer to execute image 
processing comprising the steps of: 

extracting a document block containing a 
specific image from among image data to be processed; 

recognizing character code from a character 
image within the document block; 

reconstructing the document block in a specific 

shape; and 

laying out character code data corresponding to 
the recognized character code within the reconstructed 
document block. 



BRIEF DESCRIPTION OF THE DRAWINGS 
These and other objects and features of the 
present invention will become clear from the following 
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description taken in conjunction with the preferred 
embodiments thereof with reference to the accompanying 
drawings, in which: 

FIG. 1 is a block diagram briefly showing the 
structure of an image processing device of an embodiment 
of the present invention; 

FIG. 2 is a flow chart showing the sequence of 
the document, character, and photographic image data 
generation process; 

FIG. 3 is a flow chart showing the sequence of 
the reconstruction process of the document block and 
St character code data; 

* FIG. 4 shows pre-scan image data; 

gj FIG. 5 shows document image data; 

||5 FIG. 6 shows character image data; 

P FIG. 7 shows photographic image data; 

FIG. 8 shows rectangular vector data; 

FIG. 9 shows corrected rectangular vector data; 

FIG. 10 illustrates the method for arranging 
2 0 data within the rectangular vector data; 

FIG. 11 shows completed document data; and 

FIG. 12 illustrates processing when a plurality 
of document blocks exist. 

25 DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
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The embodiments of the present invention are 
described hereinafter with reference to the accompanying 
drawings . 

FIG. 1 is a block diagram briefly showing the 
structure of an embodiment of the image processing device 
of the present invention. 

Image processing device 100 has a filing 
function for reading a document such as, for example, a 
newspaper, magazine or the like, extracting necessary 
document data from the obtained image data, and storing 
the data as an electronic file. 

The image processing device 100 is provided 
with a CPU 110, ROM 120, RAM 130, operation unit 140, 
hard disk 150, recording media drive 160, ASIC 170, and 
scanner engine 180. 

The CPU 110 controls the entire image 
processing device 100 in accordance with a program. 

The ROM 12 0 stores the control program and data. 
An image processing program described later is stored in 
the ROM 12 0. 

The RAM 130 has an area for temporarily storing 
data and programs, e.g., an area for temporarily storing 
document image data within a document block (document 
region) in the image data. 
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The operation unit 140, although not shown in 
the drawing, has a touch panel display, input keys or the 
like. 

The hard disk 150 can store an operating system, 
and various types of application programs. The hard disk 
150 also may store generated electronic files. 

The recording media drive 160 can read and 
write various types of data such as an electronic file on 
various types of removable recording media (e.g., 
flexible disk, MO disk and the like) . 

ASIC 170 is provided with a mark detecting 
section 171, image processing section 172, and area 
discriminating section 173. The mark detecting section 
171 detects a mark representing the position of a 
specific document block in the image data. The image 
processing section 172 executes specific image processing 
such as removing noise from the document image data 
within the document block. The area discriminating 
section 173 generates character image data and 
photographic image data from the document image data. 

The scanner engine 180 obtains image data by 
reading a document. 

The sequence of the document, character, and 
photographic image data generating process is described 
below using the flow chart of FIG. 2. The content of the 



flow cart of FIG . 2 is stored as a program in ROM 120, 
and is executed by the CPU 110. 

First, in step S101, the start of a pre-scan of 
a document is specified. In this way the scanner engine 
5 180 optically reads a document comprising the entire 

surface of a single page, e.g., a newspaper, magazine or 
the like. The scanner engine 180 executes a pre-scan for 
an approximate reading of a document, and executes a main 
scan for detailed reading of the document. Pre-scan 

±0 image data output from the scanner engine 180 are input 

7* to the mark detecting section 171. 

In step S102, a determination is made as to 
whether or not a mark has been detected in the pre-scan 
image data. At this time, the mark detecting section 171 

15 detects a mark 12 comprising a thick-line frame in the 

pre-scan image data 10 as shown in FIG. 4. When a mark is 
not detected (step S102: NO), the process of step S103 is 
executed, whereas when a mark is detected (step S102: 
YES), the process of step S104 is executed. This mark is 

20 added to the document by the user beforehand using a 

marker such as a felt-tip pen or the like. The user 
marks at least an area containing a headline and body 
text corresponding to the headline. 

In step S103, a screen inviting the user to 

25 attach a mark to the document is displayed on the touch 
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panel display of the operation unit 140. In this case 
the user appends a mark on the document, and the document 
block extraction process is again executed. 

On the other hand, in step S104, the coordinate 
value of each apex positioned in the angles or corners of 
the document block 14 circumscribed by the mark 12 
detected by the mark detecting section 171 is transmitted 
to the scanner engine 180. That is, in the present 
embodiment, an area specified by the user is extracted as 
a document block 14 to be processed. However, the method 
for extracting the document block 14 is not limited to 
this method. For example, an area containing at least a 
headline and body text corresponding to this headline may 
be automatically extracted as a document block 14 based 
on an image detected by well-known methods for detecting 
a character image equivalent to a headline, a character 
image equivalent to the body text, image equivalent to 
ruled lines and the like. 

In step S105, the start of the main scan of the 
document is specified. At this time, the scanner engine 
180 sets the range specified by the coordinate values of 
each apex fed back by the mark detecting section 171 as 
the range of the main scan, and executes the main scan. 
In this way the document block 14 is extracted. The main 
scan image data within the document block 14 output from 



the scanner engine 180 are input to the image processing 
section 172. 

In step S106, various image processing is 
executed by the image processing section 172. 
5 Specifically, the image processing section 172 subjects 

the main scan image data to image processing such as 
noise elimination, skew correction, vertical/horizontal 
detection, character enhancement and the like to obtain 
document image data 16 (refer to FIG. 5) . The document 

SO image data 16 output from the image processing section 

172 are input to the area discriminating section 173. 

T- In step S107, an area discrimination process is 

executed by the area discriminating section 173. 

j= Specifically, the area discriminating section 173 

■2r5 discriminates a character image area 18 containing a 

character image, and a ■ photographic image area 20 
containing a photographic image (including patterns) . 
Since the area discrimination method is a well-known art, 
details are abbreviated. Further, the area 

20- discriminating section 173 extracts a character image 

area 18 to generate character image data 22 (refer to FIG; 
6) , and extracts photographic image area 20 to generate 
photographic image data 24 (refer to FIG. 7) . 

In step SI 08, the document image data 16, 

25 character image data 22, and photographic image data 24 
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output from the area discriminating section 173 are 
stored in the RAM 130. When photographic image data 24 
are not present, only the document image data 16 are 
stored in RAM 130. 
5 The sequence of the reconstruction process for 

the document block and character code data is described 
below using the flow chart of FIG. 3. The content of the 
flow chart of FIG. 3 is stored as a program in ROM 120, 
O and is executed by the CPU 110. 

€i0 In step S201, the surface area is calculated 

=E ■ - for a document block 14 in which the document image data 
Ni .: 16 are arranged. Specifically, the surface area of the 
5. document block 14 is calculated from the total number of 

Cfl dots of the document image data 16 stored in RAM 130, and 

0j5 the resolution (dpi) of the scanner engine 180 set 

H- beforehand. The surface area of the document block 14 

also can be calculated from the coordinate values of each 
apex positioned in the angles and corners of the document 
block 14 . 

20 In step S202, the document block 14 is 

reconstructed in a specific shape. Specifically, 
rectangular vector data 26 (refer to FIG. 8) of specific 
aspect ratio and having the same surface area as the 
document block 14 are generated as a reconstructed 

25 document block. The specific aspect ratio may be set, 
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for example, at a horizontal dimension A to a vertical 
dimension B identical to A4 size, i.e., 210:297. The 
reconstructed specific shape is not limited to 
rectangular, since optional shapes may be used insofar as 
the shape is coherent and easily read. The generated 
rectangular vector data 26 are stored in RAM 130. 

In step S203, the character image data 
(document image data) stored in RAM 130 is subjected to a 
character recognition process to obtain character code 
data. The character code data includes the font size. 
The character recognition process also can be executed 
before generating the rectangular vector data.- 

In step S204, the headline and body text are 
discriminated in the character code data. The frequency 
of appearance is calculated for each font size in the 
character code data, and data having a font size greater 
than a specific value and a frequency less than a 
specific value is discriminated as the character code 
data of the headline, and text data having a font size 
less than the specific value and a frequency greater than 
the specific value is discriminated as the character code 
data of the body text. Moreover, the method of 
discriminating the headline and body text is not limited 
to this method. 
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In the character recognition process, format 
paragraph ends may be identified by appending a carriage 
return code directly after a period symbol [ . ] when one 
or more spaces occur after a period symbol [ . ] positioned 
5 at the end of a line of character code data of the body 

text. 

In step S205, the height of one column (when 
body text is written vertically) of the multiple columns 
O formed within the document block 14 is calculated. 

C-0 Although the present example describes body text written 

4* vertically, the present invention also is applicable to 

^ body text written horizontally. The height of one column 

^ of the multiple columns can be calculated, for example, 

© from the font size of the obtained character code data, 

|fe and the number of characters per column of the multiple 

M= columns . 

In step S206, the vertical dimension B of the 
rectangular vector data 2 6 is corrected to a length of an 
approximate natural integer multiple of the height C of 

20 one column of the multiple columns 28 (refer to FIG. 9) . 

In this way rectangular vector data 2 6 are obtained 
without wasteful white space while maintaining the height 
of one column of the multiple columns in the document. 
At this time the corrected vertical dimension is 

25 corrected to a value nearest the vertical dimension of 
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the initial rectangular vector data 2 6 shown in FIG. 8. 
Then, the horizontal dimension A is corrected in 
accordance with the correction of the vertical dimension 
B so as to maintain the surface area of the initial 
rectangular vector data 26. The corrected rectangular 
vector data 26 are again stored in RAM 130. 

In step S207, a determination is made as to 
whether or not the character code data of the headline 
can be arranged within the corrected rectangular vector 
data 26. Specifically, the horizontal length and the 
vertical length of the headline character code data are 
compared to the horizontal dimension A and the vertical 
dimension B of the corrected rectangular vector data 26. 
When the character code data of the headline protrude 
from the rectangular vector data 2 6 (step S2 07: NO), the 
process of step S208 is executed, whereas when the 
character code data of the headline can be arranged 
within the rectangular vector data 26 (step S207: YES), 
the process of step S209 is executed. 

In step S208, the rectangular vector data 26 
are again corrected in accordance with the size of the 
headline character code data, and again stored in RAM 130. 
At this time the rectangular vector data 2 6 is enlarged 
only in the dimension in the direction (vertical or 



horizontal) in which the headline character code data 
protrudes . 

On the other hand, in step S2 09, a 
determination is made as to whether or not photographic 
5 image data 24 are stored in RAM 130. When photographic 

image data 24 are present (step S209: YES), the process 
of step S210 is executed, whereas when photographic image 
data 24 are not stored in RAM 130 (step S209: NO), the 
^ process of step S214 is executed. 

Z0 In step S210, a determination is made as to 

= whether or not the photographic image area 2 0 in the- 

photographic image data 24 can be arranged within the 
= _■ corrected rectangular vector data 26. Specifically, the 

Z~ horizontal length and vertical length of the photographic 

15 image area 2 0 are respectively compared to the horizontal 

dimension A and vertical dimension B of the corrected 
rectangular vector data 26. When the photographic image 
area 20 protrudes from the rectangular vector data 2 6 
(step S210: NO), the process of step S211 is executed, 
20 whereas when the photographic image area 20 can be 

arranged within the rectangular vector data 2 6 (step 
S210: YES), the process of step S212 is executed. 

In step S211, the rectangular vector data 26 
are again corrected in accordance with the size of the 
25 photographic image area 20, and again stored in RAM 130. 
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At this time the rectangular vector data 2 6 is enlarged 
only in the dimension in the direction (vertical or 
horizontal) in which the photographic image area 20 
protrudes . 

5 On the other hand, in step 3212, the position 

of the photographic image area 2 0 in the photographic 
image data 24 is calculated. Specif ically, the top right 
position of the photographic image data 24 (identical to 
the top right point of the document image data 16) is set 
ft) as a standard, and the coordinates (distance r, direction 

If 0) of the top right-most position of the photographic 

image area 20 are calculated (refer to FIG. 7) . 
T In step S213, first, the photographic image 

go area 2 0 is arranged within the rectangular vector data 2 6 

li|5 in accordance with the calculated position, as shown in 

FIG. 10. In this way the layout of the original document 
is maintained to a degree. 

In step S214, the headline character code data 
30 are arranged at a specific position, specifically at 
2 0 the top right, within the rectangular vector data 26. In 

this way the readability is improved. When the body text 
is written horizontally, the headline character code data 
30 is arranged at the top left. 

In step S215, the body text character code data 
25 32 are arranged within the rectangular vector data 26. 
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At this time the body text character code data 32 are 
sequentially arranged from top right to bottom left in 
the white space corresponding to the multiple columns 2 8 
outside the areas in which the photographic image area 20 
and the headline character code data 30 are arranged. 

In step S216, the document data 34 (refer to 
FIG. 11) which has been completed by arranging the 
headline character code data 30, the body text character 
code data 32, and the photographic image area 20 within 
the rectangular vector data 2 6, are stored on hard disk 
150 as an electronic file. The document data 34 also may 
be saved on a flexible disk or the like via the recording 
media drive 160. 

According to the above embodiment, a document 
block 14 is extracted from among image data obtained by 
optically reading the image of a document such as a 
newspaper, magazine or the like using a scanner engine 
180, character code is recognized from the character 
image within the document block 14, rectangular vector 
data 26 are generated to reconstruct the shape of the 
document block 14, and the character code data 
corresponding to the recognized character code are laid 
out within the rectangular vector data 26. Accordingly, 
specific document data are extracted from among image 
data obtained by reading a document such as a newspaper, 
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magazine or the like, such that the document data can be 
readily obtained which are easily readable and 
effectively adhered without waste in, for example, a file 
region of definite form. 
5 Although the present invention has been fully 

described in connection with the preferred embodiments 
thereof with reference to the accompanying drawings, it 
is to be noted that various changes and modifications are 
« apparent to those skilled in the art. Such changes and 

10 modifications are to be understood as being included 

within the scope of the present invention as defined- by 
~l= the appended claims unless they depart therefrom. 

For example, although the present invention has 
; been described in terms of a single document block 14 

±;5 circumscribed by a mark 12 in pre-scan image data 10 of a 

*~ read document in the above embodiments, it is to be 

understood that the present invention is not limited to 
this arrangement. The present invention also is 
applicable to instances of a plurality of document blocks., 
20 as shown in FIG. 12 (A) . In this case the surface area of 

the document block is determined as the sum of the 
surface areas of each document block 14. Then, the 
headline character code data 30, the body text character 
code data 32 and the like of each document block are, for 
25 example, sequentially arranged in the white space of the 
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rectangular vector data 26 from the top right to the 

bottom left (when the main text is written vertically) . 

This layout method is not specifcially limited. 

According to this example, a needed document can be 
5 collected in a single document even when it exists in a 

plurality of division within a document, thereby the 

document data can be easily obtained which are readable 

and more effectively adhered. 
Q In the above embodiments, the circumscription 

-jfcO of a document block 14 is specified by a mark 12 and 

£ extracted, however, all document blocks may be 

automatically extracted from image data of a read 
= document comprising the entirety of one page such as a 

K newspaper, magazine or the like. 

= J.5 The image processing device also may have, in 

™ addition to the structure shown in FIG. 1, an interface 

for sending and receiving data between the image 
processing device and other inf ormaiton devices. In this 
way a generated document data 34 (refer to FIG. 11) may be 
20 transmitted to another informaiton device such as a 

computer, printer or the like. 

The image processing device also may have, in 
addition to the structure shown in FIG. 1, a printer 
engine for printing data on recoridng substrate such as 
25 cut paper, OHP transparencies, roll paper and the like. 
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In this way a generated document data 34 may be printed 
on recording substrate. 

The components comprising the image processing 
device and the image processing method of the present 
invention may be realized by special hardware circuits or 
programmed computer. Furthermore, when the present 
invention is realized by programmed computer, the 
programs which operate the computer may be provided on a 
computer-readable recording medium (e.g., floppy disk, 
CD-ROM and the like) . In this case a program recorded on 
a computer-readable recording medium normally is 
transferred to and recorded on a hard disk. The program 
also may be provided as independent applicaiton soiftware. 
Furthermore, the program may be incorporated in the 
software of the computer as one function of the computer. 



